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Abstract 

Now the Gibbs estimators are used to solve many problems in computer vision, 
optimal control, Bayesian statistics etc. Their determination is equivalent to mini- 
mization of corresponding energy functions. Therefore, the real use of the estimators 
, is restricted by opportunity to compute or evaluate satisfactory values of the energy 

Cp I functions. New combinatorial techniques that have been developed for the last years 

allowed efficient employment of high dimensional Ising models with Boolean and ra- 
ff^ . tional valued variables. In addition, graph cut methods were proposed to minimize 

O ; energy functions U = Y, 9^l{x^) + Y.9^J.,v{x^, Xy) + Y, gfi,v,K{x^,x^,Xi^) of Boolean vari- 

^ ' ables and U = ^ S'dv ~ °f integer variables with submodular 

■ functions g. 

^ \ In the paper energy functions that often occur in applications (for instance, in 

estimating of images) are considered. They are suppose to be depended on variables 
taking values in totally ordered finite sets (finite sets of arbitrary numbers are among 
them). The representation of functions by Boolean polynomials are described. Fast 
I methods of minimization of submodular Boolean polynomials that correspond to the 

energy functions are developed. For that the Modified algorithm of minimization of 
submodular functions (MSFM) is proposed. Also the minimum graph cut technique is 
presented for submodular Boolean polynomials that satisfy some additional condition. 
Among such polynomials are, for instance, those with all coefficients before nonlin- 
ear monomials negative as well as submodular Boolean polynomials, which have all 
coefficients before monomials of order more than two positive. 
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1 Introduction 



Nowadays Gibbs models are successfully employed in mathematical statistics, statistical 
physics, image processing etc. Often these models allow to get satisfactory solutions of 
various problems but usually they turn out very hard to be solved efficiently. Finding of 
Gibbs estimates is equivalent to the problem of identification of minima of corresponding 
energy functions. In this paper we present fast methods of finding the Gibbs estimates for 
some models that occur in application (for instance, in image processing). 

The Simulated annealing [21 IH El UD] was one of the first method that was used to find 
the Gibbs estimates. In spite of its slowness, it remains a very popular tool to evaluate the 
estimates as well as other problems of general global optimization. To speed up computation 
of the estimator of the Ising model Grieg, Porteous and Seheult |S] in 1989 described the 
network fiow technique that allowed the exact Boolean minimizing the Ising energy function. 
In the last few years several new approaches, which enable efficient estimation jl] or even 
efficient minimizing some wide classes of functions of many Boolean or integer variables, 
have been developed El lUI- These approaches, which are based on the graph cut 
technique, demonstrate possibility to solve real practical problems including handling of 
3D data for acceptable time (for instance, segmentation of 3D 200 x 200 x 300 gray-scale 
image by the such type method took 14min. of Pentiym 4 1.6GH CPU). 

Recently the combinatorial algorithm of minimization of suhmodular functions (SFM) 
have been developed [7j. To minimize a submodular function of n variables it needs either 
0(n^ log M) (where M is some number depending on the function considered) or 0{n^) op- 
erations. Therefore, the SFM can not be immediately used for finding of high dimensional 
Gibbs estimates for its comparatively high computational expenses. We present a modifica- 
tion of the SFM that can be useful for minimization of submodular Boolean representations 
of energy functions, which occur in applications, for instance, in image processing. 

In order to use the SFM algorithm and the graph cut technique for energy functions, 
which depend on variables taking values in finite totally ordered sets, we represent the func- 
tions by Boolean polynomials. Then we develop methods that allow efficient minimization 
of submodular Boolean polynomials of special structure. Some Boolean polynomials admit 
finding parts of Boolean vectors, which minimize these polynomials, by use only parts of 
their monomials. We illustrate this property with image processing terminology. It means 
colours of far points of images estimates are independent, i.e. a change of colours of some 
image pixels in the estimate does not affect colours of other far pixels. Therefore, the 
modified SFM (MSFM) operates, first, not with an original Boolean polynomial but with 
separated its parts, trying to find part of coordinates of a solution of the original problem. 
Then the values found are set in the original polynomial and the procedure are repeated 
either with other parts of the residual polynomial or with the entire residual polynomial. At 
worst, it takes as many operations as the usual SFM and, at best, the number of operations 
is even 0(n), where n is a number of Boolean variables. 

As we mentioned above, new graph cut methods jHl E] are now an efficient tool 
to determine Gibbs estimators with energy functions U = ^(7/i(x^) + J^dfj^fi^fj-^^i^) + 

9iJ.,u,K{Xfj,, Xu, Xk) that are Boolean polynomials of the third order. We failed to represent 
an arbitrary submodular Boolean polynomial by a graph but we made it for submodular 
Boolean polynomials of the special structure that can be met in applications. The set of 
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those polynomials contains, for instance, Boolean polynomials with all nonlinear monomials 
negative (of course, such polynomials are submodular) and submodular Boolean polynomi- 
als with monomials of order three and higher positive. 



2 Representation of Functions by Boolean Polynomi- 
als 

Below the function f/(x), x = (xi, . . . , x„) is supposed to be dependent on variables that 
take values in an arbitrary finite totally ordered set IZ = {tq, ri, . . . , r^}, tq < ri < . . . < rk. 
The multiindex j = (ji, j2, • • • ,jn) is with coordinates ji G {0, 1, . . . , A;} = Nk- The function 
of integer variables 

^(j) = Uirj^,rj^,...,rjJ 

is considered instead of the initial U{x.). 

To exploit the MSF and graph cut technique we, first, represent the integer variables ji 
as the sum of ordered Boolean ones 

k 

1=1 

where G {0, 1}, Xi{l) > Xj(2) > . . . > Xi{k). By other words, we establish one-to-one 
correspondence between each variable ji and the ordered sequence of Boolean variables 
> Xj(2) > . . . > Xi{k). After we represent the function V{i) by a Boolean polynomial 
and then minimize the polynomial. 

Denote the Boolean vector x(/) = {xi{l) , X2{1) , ■ ■ ■ and for two vectors x, z say 

X > z, if Xj > i = 1 -r n (respectively, x ^ z, if there is a couple of indexes i, j such that 
Xi < Zi and Xj > Zj). Then 

^(j) = ^ (^''^^^) ' ""^^^ - ""^^^ 

For simplicity let us start with the function of two variables V{ji,j2)- It is not hard to 
check the equality 

k 

^(Jl, J2) = V{0,J2) + 5^(l^(^, J2) - V{fl - l,j2))Xl{fl), (2) 

where ordered Boolean variables Xi(/i) satisfy the equality (0). Now the expansion 

k k 

= V{0, 0) + ^(V^(/i, 0)-V{fi-l, 0))xi(/i) + 5^(1^(0, u) - V{0, V - \))x2[v) 

(1=1 u=l 

k 

+ J2 (^(/^' ^) - ^(/^ - 1> - Vif^, - 1) + F(/i - 1, 1^ - l))x,{fi)x2{iy), (3) 

li,u=l 
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can be constructed by use of equality (0) for the second variable j2 of each difference 

Vifi,j2)-Vif,-l,j2).^ 

In general case, for j G N„ we say 



A.y(j) = V{j) - V{i - e,), e, = (0, . . . , 0, 1,0,. ..,0) 



is difference derivative of the first order, and define the mixed difference derivative of higher 
orders by the recursion 

Note that for any permutation of indices vr(ji, j2, • • • ,jn) 
By analogy with 2D case 

k 

V{i) = U{0,J2, . . .,Jn) + J] Aiy(/Z, J2, . . . , Jn))Xl(/i). (4) 

Let the Gm be the projection operator onto the space of last n — m coordinate, i.e. 

Cmj = (0, . . . , 0, im+l) im+25 • • • ) jn)- 

Applying formula (jl)) recursively m times to each its item we get the equality 

k 

l<l<m (1=1 
k 

l<'l<'2— ™ Ml 1/^2=1 
k 

+ ^ Ai,2,...,mV^(G'mj + /xiei + /X2e2 + . . . + /imem)a;i(An)a;2(Ai2) • • -aJml/Xm) 

Ml.-,/im=l 

Therefore, the following statement is valid. 

Proposition 1. The expansion of V{j) into a polynomial in ordered Boolean variables 
x(l) > x(2) > . . . > x(A;) is of the form 

y(j) = P^(x(l),x(2)...,x(fc)) 

n k / m \ m 

= v{o) + j2 E E ^iu...uviY.MJi[xiM- (5) 

m=l I < li < . . . < < n ^ti,...,^im=l \3<=l / }<=1 
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The expansion (jSj) does not allow to turn immediately to the problem of the Boolean 
minimization because of ordered variables. To avoid this disadvantage we will use another 
polynomial 

n k 

Pv (x(l), x(2) . . . , x(A;)) = Pv (x(l), x(2) . . . , x(A;)) + C ^ ^^(2:^(0 " " 

fi=l 1=2 

for sufficiently large constant C > 0. This polynomial satisfy the following properties. 

Proposition 2. The inequality Py < Py holds true. Any collection of Boolean vectors 
u* = (x*(l),x*(2) . . . ,x*(A;)) that minimizes Py 

u* = argmin (x(l), x(2) . . . , x(A;)) 

x(l),x(2),...,x(fc) 

turns to be ordered (non-increasing^. This collection minimizes the polynomial Py, and, 
therefore, the integer vector j* = X]i=i^*(0 minimizes the original function V{j). 

Therefore, instead of minimization of V{j) we can consider the problem of Boolean 
minimization of the polynomial 

n m 

Py{u) = ao + ^ ^ aii,/2,...,Zm n^'i' dim(u) = kn. 

m=l 1 < li < . . . < Im < n i=l 

Because of the representation of functions V{j) by Boolean polynomials developed we con- 
sider the problem of minimization of general Boolean polynomials -P(x). 

3 Submodular Boolean Polynomials 

For two Boolean vectors x, y let us denote by x V y (respectively, by x A y) the vector with 
coordinates ma.x{xi,yi) (respectively, with coordinates min(xj, 

Definition 3. A function f is called submodular if for any Boolean vectors x, y it satisfies 
the inequality 

/(xAy) + /(xVy)</(x) + /(y). 

Let the set V = {1, • • • ,n}, and for any V G V the set = V\D. Let the vector 
xx) with coordinated Xi, i E V he the restriction of x to the set T>, the abbreviation 
P{^v) = P{^j,,Ovc). 

Below we always suppose P{0) = 0. Therefore, P is of the form 

n m 
m=l 1 < /i < . . . < < n i=l 

For any z, j G V a polynomial -P(x) can be written in the form 

P(x) = P(x{jj}c) + XiPi(x{ij}c) + XjPji^X^ijyc) + XjXj Pj j (x{j j}c) (7) 
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or in the form 

P(x) = g(x) + L(x), 

where Q(x) is the polynomial consisting of monomials of degree 2 and higher, and L{'x) is 
a linear polynomial. It also can be represented as 

P(x) = F{P}(x) + PM, VCV, 

where each monomial of P{P}(x) contains at least one variable x,, i (zT) (and, in general, 
it depends on xx)c), and F(x75c) is independent of xp. The following useful properties can 
be easily veryfied. 

Proposition 4. a) A Boolean polynomial P(x) is suhmodular if and only if for any vector 
X and any couple of indices i,j E M the polynomials Pij{'x.^ijjc) < 0. 

b) In order to Boolean polynomial P(x) be submodular it is necessary and sufficient its 
nonlinear part Q{x.) will be submodular. 

c) //P(x) is submodular, then the polynomial Q{x.) is nonin creasing, in the sense that 
Qiy) ^ for any ordered pair of vectors y > x. Therefore, argmin^(5(x) = ly. 

d) //P(x) is submodular, then P{P}(x) is submodular polynomial in Xx) for each fixed 
vector xx>c . 

The sentences a), b) and d) are deduced directly from Definition |21 To prove c) let us 
consider a couple of vectors: x with Xi = and y = x + ej and inequality 

Q(Ov) + Q(y) < g(e,) + g(x) 

that follows from Definition |21 Since g(Ov) = Qi^i) = we have Qiy) < g(x). To prove 
c) in general case it is enough to repeat the last inequality successively. 

We need two properties of polynomials P{P}(x). The first one is rather clear. 

Proposition 5. //x^ minimizes the polynomial P{T)}(x) for some fixed xjjc, then for any 
set £ C T) the restriction x^ minimizes the polynomial P{£^}(x£, x^^^, x^^c) with respect 
to Xf, and vice versa, if the vector Sc^ minimizes P{£^}(x£-, x^^^^, Xx)c), then the vector 
^vxs) ™nimizes P{P}(xx), x^jc). with respect to xx>. 

The second property concerns monotonic (in some sense) dependence of the solution 
x^ = argmin^j, P{2^}(xx), Xx)c) on boundary vector x^^c for a submodular polynomial P. 
Let 

A^(xx,c) = |argminP{r'}(x75,Xpc)| 

be the set of minima of the polynomial P{T>} for fixed x^jc. It can be shown that for two 
ordered boundary vectors :xix><^<zx>c can exist either unordered x^ ^ or non-increasing 
> z'jy solutions, but for any x^ there exists such that x^ < z^, and vice versa, for 
any z^ there is x^ satisfying the same inequality. 
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Theorem 6. Let a polynomial P he suhmodular. Lei /or Xx)c<zx)c solutions x' G 7V/i(xx)c) 
and z' G A^(zx)c) (io not satisfy the inequality x^ < z^. 

Then for the maximal nonempty set of the inverse order £ = {i & V \ x[ = 1, z[ = Q}, 
such that If = x^ > z^ = 0^:, the vector (z^,x^^^) = (0£:,x^^^) G 7W(xx)c) as we// as 

{'^£^'^'v\s) = (l£-'Z^\f) G A<(zpc) anc/, therefore, (z^,x^^^) < (x^,z^^^). 

Proof. For boundary vectors :s.f)c<Zj)c suppose the condition x^ < z^ is not vahd, i.e. 
either x^ > z^ or x^ ^ z^. In this case the set £^ = G | = 1, z'^ = 0} is nonempty. 
Let us consider the chain of relations. The inequahty 

P{S}{z'„ z'^^„ z^c) < P{S}{^'„ z'^^„ z^c) (8) 

holds true since the vector z^ minimizes P{£}{ ■ ,z^^^,zx)c) (see, Proposition E)) . The 
inequality 

P{S}{^'„z'^\„Zr,c) < P{f}(x^,X^\f,Xpc) (9) 

will be proved later. The relation 

P{S}{^'„^'.r,\e,^vc) < P{^}(z^,x^\f,xpc) (10) 
is verified the same way as (jH)). Equalities 

P{8}{z',y^\„^r,c) = P{S}{z'„z'^\„°Zt,c) =0 (11) 

are valid because z^ = O^. Therefore, relations (jH|)- (fTT|) are, actually, equalities, and the 
claim of the Theorem can be derived from Proposition El 

Thus, the Theorem will be proved if we ground the inequality Q. For that we need 
additional notations. One-element sets {j} E V will be denoted by j. Instead of £^ U {j} 
and we will use abbreviations S + j and S — j. For arbitrary sets A,B,C G V 

we denote by P{A,B\C}{x.) polynomials that consist of monomials of P(x) with variables 
Xi, i E A, Xj, j E B but without Xk, k E C. 

o o 

First, consider Boolean vectors that for some j G satisfy the equation x^-c -j-ej =Z£c 
and the polynomial P{£ + j}(xf+j, Xfc_j), which is a submodular function of x^+j. Let 
us write the condition of submodularity of the polynomial for vectors a^+j = (0^, Ij) and 
hg^j = (x£;,Oj) using the representation 

F{^+j}(Xf+„XfC_,) =P{^|r}(Xf)+P{^,j}(Xf+,,XfC.,) + 

P{£,£^ - j\j}{x£,^£c_j) + P{j,£^ - j\£}{xj,^£c_j) + ajXj. 
Since a^^j A h^^j = O^^j and a^+j V h^^j = (x^:, Ij), it looks as 

P{£\£'}M + P{£,j}{^£, Ij, Xfc„,)+ 

P{£,£'' - j|j}(xf ,xfc_,) + P{j,£'= - j|^}(l„xfc_,) + aj < 
P{j,£'-j\£}{l„^ec^,) + a,+ 

P{£\£''}M + P{£,j}{^e,Oj,^ec-j)+P{£,£''-j\j}{^e,^£c_^). 
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Therefore, 

P{f,j}(x^,l,-,X£c_,) < P{^,j}(x£,0,-,x^c_,) = 0. (12) 

But 

P{f}(xf:,X£c) = P{£:|r}(xf:) + P{£:,j}(x^,x,,x^c_,) + P{f,£:^-j1j}(x£,x^c„,) (13) 
Inequality (fT^ . equation (fT^ and condition :K£c_j=Z£c_j allows to conclude 

P{^}(x^,x^c)>P{^}(x^,z^c). (14) 

o o 

To prove p4j] for an arbitrary pair X£-c<Z£-c it is enough to use monotone increasing sequence 

o o o o 

of boundary vectors X£-c<X£-c i< . . . Kx^c ^Kz^c that differ with one coordinate only. □ 
Theorem IHl allows to describe some properties of the set of solutions 

A4 = |argminP(x)|> . 

o o 

Corollary 7. Let^gc<z^c he any ordered pair of vectors, then: 

(i) for any x^ G A^(xx)c) there exists a solution G Ai{zDc) such that x^ < z^. 
(a) For any z^ G Ai(zj,c) there exists a solution x^ G Ai('Xj)c) such that x^ < z^. 
(Hi) The sets A4{'X'pc) and M.{zx,c) have minimal x!^,z^j) and maximal ^i'^.z'j^ elements, 
(iv) The minimal and maximal elements are ordered, i.e. ]dp < z^ and x^ < z^. 

The sentences (i) and (ii) follow directly from Theorem IHl Sentence (Hi) is deduced 
from Theorem ini for '^^c=z^c . And, at last, (zw) can be derived from (z,m) and definitions 
of minimal and maximal elements. 



4 Modified SFM Algorithm 

The main idea of the Modified SFM algorithm (MSFM) is to determine (if possible) parts 
of a solution x* = by use separate parts Pj'E'iKxx)., xx)?), {Vi[jVj = 0, f]T>i = V) 
of the original polynomial P(x). For this purpose monotone dependence of local solutions 
x^. (in the sense of Corollary Ej) on the frontier vector x^c can be exploited. If the vector 
x^. G A4{lgc) has some coordinates Xj = 0, then there is a solution x* E Ai with the same 
coordinates x* = 0. Similarly, if the solution x^^ G Ai^O^c) is with coordinates x'j = 1, 
then there is a solution x* G that has x* = 1. 

The strategy developed leads to success not for all Boolean polynomials, but it is useful 
for polynomials that have local dependence of variables. Such polynomials are quite often 
used in applications, for instance, in Bayesian estimation or image processing. 

Describe the MSFM in details. Partition the set V by suitable sets 'Pj(l), . . . ,©^^(1) 
{Vi{l)[jVj{l) = 0, rWi{l) = V). It follows from Proposition El that x^.^. G Al(x^c^J, 
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i.e. x^.(i) minimizes the polynomial P{I'j}(xx)., x^.j^-,^-|). Corollary [7| substantiates existence 
of triple of solutions 

xo,c.(i) e ;V/l(055C(i)), xi,2,.(i) e A^(li,c(i)) and x'^.(i) G A^(x^c(i)) 

such that xo,x'^(i) < x^.^-^-j < xi_x>j(i) (the vectors can be found by the usual SFM algorithm). 
Therefore, solutions x^.^^^-^ have coordinates x'^^.i^i^j = for sets of indices Bi{l) = {j E 
T>i{l) I = 0} as well as they have coordinates = 1 for sets of indices 

Wi{l) = {j e I a;o,7,,(i),, = 1}. 

k-t 

Check the set 7?.(1) = IJi=i(^i(l) U ^ii^))- If it is empty we either try another partition 
or start the standard SFM for the original polynomial -P(x). 

If 7^.(1) 7^ 0, we identified the part of the solution x^^j.-,^^ = y^'-m^iy Set V(2) = V\71{1) 
and consider the reduced problem of identification of 

Xv(2) = argminP(xv(2),x;j(i)). 

XV(2) 

Partition V(2) = \Ji^iVi{2), Vi{2) f]Vj{2) = and estimate x^.(2) by solutions 

Xo,Dj(2) e A^(0pC(2)\7^(i),X^^(l)) and Xi^x)j(2) e A^(l55C(2)\7^(i),X^(y), 

using sets i3i(2) = {j e Vi{2) \ xi,d,(2)j = 0} and W.(2) = {j E V,{2) \ xo,75.(2),j = 
1} (Corollary m grounds existence of triple of vectors Xo_d^(2) < ^ti-{2) — -^i.^jCs) )■ The 
algorithm is iterated at high levels until the problem will be completely solved or until 
7l{l) 7^ 0. In the last case the standard SFM is after applied. Sizes of partitioning sets 
T>i{l) should be sufficient to have 7^(/) nonempty but not very large to have small enough 
number of operations. It is unlikely possible to estimate them in general case since for some 
polynomials the MSFM is reduced to the usual SFM for any partition. However, in many 
applied problems of image processing or Bayesian estimation the finding suitable sizes of 
is not difficult. In these problems energy functions usually have lack of long distance 
interactions (or phase transitions). It means solutions x.^ for the original image and an 
image, which coincides with the original on sufficiently large set P C V and differs from it 
on V^, are approximately equal. By other words, for real applications estimates of image 
details do not depend on its surrounding objects. In this case the MSFM is preferable to 
the standard SFM. The number operation required can be here even 0{n). 

The identification of ^b-(i)\j-v\;-(i) can be done in concurrent mode. The number of levels 
depend on the polynomial P. Usually, it is enough two or three ones. Formal description 
of the MSFM is rather simple. It is placed in Fig.l. 
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MSFM(P): 



Initialization: 

m < — 1 the enumerator of levels 

L < — a number of levels 

V(l) ^ V 

{X'i(l)} < — a partition of V 

{xo,©i(i),Xi,c.(i)} < — such that Xo,i?.(i) < ^i,Vi{i), by the SFM 

{Bi{l),Wi{l)} < — coordinates of the part of the solution found 

^(1) ^ u£i(m(i)u^i(i)) 



If 7^(l) = then use the SFM, Stop 



^v,(i)j ^ for J ei3i(l) 

^v,ii),j ^ 1 for j e 
While m < L do 

m < — m + 1 

V(m) < — V(m- l)\7^(m- 1) 

{Pj(m)} < — partition of V(m) 

Pii^Vim)) * — the residual polynomial P(xy(^),x^c(^^P 

{^,Vi{m),^i,Vi(m)} < — such that xo,2?.(„) < xi,i,.(^), by the SFM 

for PKxv(m)) 

{Bi{m), yVi{m)} < — coordinates of the part of the solution found 

If US {yVi{m)\JBi{m)) = then use the SFM, Stop 

nm) ^ 7^(m-l)U(U£i(>^^HU^^M)) 

^v,im),j ^ for J G^.(m) 

^vAm\i ^ 1 forje>Vi(m) 



If n{L) ^ then use the SFM 

Return x* 

End 



Fig.l 
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5 Representability of Some Submodular Polynomials 
by Graphs 

Recently Kolmogorov&Zabih [S] have described some class of functions V^(x) in Boolean 
variables x ) that permit minimization by the graph cut technique. For 

convenience we reformulate and then use their results in terms of Boolean polynomials. 
We also represent some submodular Boolean polynomials by graphs to make possible their 
efficient minimization via graph gut technique (recall that finding the minimum graph cut 
requires 0{N^) operations on number of nodes). 

Let the network F = {S, A) consist of + 2 numbered nodes S = {0, . . . , + 1}, where 
s = is the source, t = + 1 is the sink and V = {1, . . . , A^} are usual nodes. The set of 
directed arcs is ^ = : S}. The capacities of arcs are denoted by dij > 0. 

The cut (we use this term as abbreviation of the term s-t cut) is a partition of the set S 
by two disjoint sets W, B such that s G VF and t E B. For any cut {W, B) nodes i E W will 
be labeled by = 1 and nodes i E B - hj Zi = so that there is one-to-one map between 
cuts and Boolean vectors z = {zi, . . . , zn). 

5.1 Graph representability 

In jHj the following definition of graph representability is given. 

Definition 8. A function V(x) of n Boolean variables is called graph representable if there 
exists a network F = {S,A) with number of nodes N > n and the cost function C(z) such 
that for some constant c 

V'(x) = min C{xi, . . . , x^, z^+i, ■ ■ ■ , zn) + c. 

Zn+ 1, ■ ■ ■ , 

Costs of the graph cuts C(z) are equal to values of the quadratic Boolean polynomial 
p(z) of the form (see, 

C{z) = p{z) = ^ds^i{l - Zi) + ^di^tZi+ ^ dij{zi - Zj)zi+ c, dij, c > 0. (15) 

Therefore, we can give an equivalent definition of the graph representability. 

Definition 9. We say a function V{x.) of n Boolean variables can be represented by graph 
if there exists a quadratic polynomial p(z) of N {N > n) Boolean variables of the form 
/ li,5|) . which satisfies the equality 

y(x) = min p(xi,...,Xn,2;n+i,---,^Ar) + c. 

2n + 1 1 • ■ • I ^JV 

The class JF^ of Boolean functions that are often used in applications and that are, 
actually, quadratic Boolean polynomials 

e J"^ y(x) = Pv'(x) = ^ aixi+ ^ 0.1^,12^1^X1^+ c. (16) 

l<l<n l<h<l2<n 
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was considered in [HI . The characterization of the graph represent abihty of y G T"^ in terms 
of Boolean polynomials has been done. It can be used, in particular, to minimize functions 
of integer variables. 

Theorem 10. A junction V G JF^ is graph representable if and only if all quadratic coef- 
ficients of its polynomial representation -Pv(x) are nonpositive, i.e. ai-^j^ < 0. Therefore, 
in order to function V G J^^ be graph representable it is necessary an sufficient it will be 
submodular. 

The sufficiency of the Theorem follows immediately from Definition IHl and Proposi- 
tion |3] a). The necessity will be proven below for more general case. The following result 
has been proven in |^ by graph cut technique. 

Theorem 11. In order to polynomial -P(x) of degree m can be represented by a graph it 
should be submodular. 

Proof. To prove this sentence we use representation ((7j) of the polynomial -P(x) at page 1^1 
For arbitrary couple of indices i,j let us fixed the vector Xjjjjc Since -P(x) is graph 
representable the quadratic polynomial XjXjPjj(x{ij}c) of two variables graph rep- 

resentable, as well. That is 

XiXjPi^j{yi{i^jY) = ™^ P{Xl,X2, Zn+l, ...,Zn)+ c 

+ 1 1 ■ • • 1 ^JV 

(in general, the quadratic submodular polynomial p and c depend on ^{ij}c but it is 
fixed now). Suppose Pjj(x|jj}c) > 0. Then p has at least two values of the argument 
(xi, X2, -Zn+i, . . . , zn) that minimize the polynomial globally. One of them is with coordi- 
nates (0, 1, . . .), and the other is of the form (1,0, . . .). Because of submodularity of p it 
follows from Corollary [7| (i) there exists the third vector, which minimizes p globally. This 
vector has coordinates (1,1,...). The existence of such the vector contradicts with the 
assumption Pjj(x{jj}c) > 0. Therefore, Pjj(x{jj}c) < for any couple of indices i,j and 
vector X{i jjc and the sentence of the Theorem follows from Corollary |3] a) . □ 



5.2 Graph represtability of some subset of submodular polyno- 
mials 

The submodularity is necessary condition in order to a Boolean polynomial can be rep- 
resented by a graph. Below we prove representability by graphs of some subset Vsuf 
of submodular polynomials. To define this subset we use formula ((Tj) at page El Let 
ki,...,imihj) = ai^,...,i,...,j,...,im^ (h < . . . < i < . . . < j < . . . < Im, m = ^ n - 2) he coeffi- 
cients of Pjj(x{jj}c) and 6^ im^'^^j) be the positive ones (note that b{i,j) = aij < 0). 
Let the set of polynomials 



n-2 

Vsuf =<P ' 



m- 



=1 li<l2<---<lr 



This set is a proper subset of all submodular polynomials Vsubmod (see Proposition lU a)). It 
contains, for instance, the set JF™ = {P | ai^^i^^,,,^ij^ < 0, A; = 2^m} of Boolean polynomials 



12 



with all coefficients of nonlinear monomials nonpositive and the set JF™ = |P | P g 
Vsuhmod, cii-^,i2,-,if. > 0, A; = 3-rm} of submodular polynomials with all coefficients of order 
more than two nonnegative. 

Remark 1. Note the condition 

n-2 

m=l li<l2<...<lm 

is necessary and sufficient in order to the polynomial, which has all coefficients of order 
more than two positive (i.e. P E Vgubmod), be submodular. 

Polynomials of those types can be used, for instance, in image processing for recognition 
of images or Bayesian estimation. To prove representability of P G Vsuf by graphs we need 
two lemmas, which give a constructive tool to build a corresponding graph as well. 

Lemma 12. For any natural m and real a < the Boolean polynomial axiX2 ■ ■ ■ Xm is 

The graph that represents drawn in Fig. 2. 

It needs one additional node z. The corresponding poly- 
nomial is p{xi, X2, . . . , Xm, z) = —a Yl^li^ ~ ^i)^ + '^-2- 
If at least one Xi = 0, then min^ p{xi, X2, ■ ■ ■ , Xm, z) = 
otherwise minzp{l, I, ... ,1, z) = a. Hence, axiX2 ■ ■ - Xm 
= mmzp{xi,X2, . . .,Xm,z). 

The monomials of order greater than 2 with positive 
coeffisients can not be represented by graphs directly. To 
do it we should accompany those positive monomials by 
quadratic items with large enough negative coefficients. 
It requires additional nodes in graphs that represent the 
monomials. 

Lemma 13. For any natural m > 2 and a > the Boolean polynomial P{xi, X2, . . . , Xn) = 

graph representable. 

Proof. In order to prove the Lemma we construct for each m > 2 a quadratic Boolean 
polynomial that specifies graph (the examples of graphs for m = 3 -i- 6 are drawn in Fig. 3 

at page ITTj) . The number of additional variables of this polynomial is / = — - — if m is 

m — 2 

odd and is / = if m is even. It is of the form 

2 

I m I 

p{xi, ...,Xm,Zi,...,zi) = ^bj ^{zj - Xi)zj + E Cj^i (17) 

j=l i=l j=l 



graph representable. 



s . 




-a 






z 




• 








\ 


X "V 


-a\ ^3 


Xy 


X2 
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with all bj > and ej < 0, such that the minimum of p with respect to 2^1, . . . , 2;; is equal 
to corresponding values of P. Those values are 



P{X\^ X2i . . . , 



—a 



—a 



if all Xi = 0, 
if Xi^ = 1 and Xi 



if X 



X 



1 and X. 



*fc + 1 



a, if Xi 



1. 



Let the vector 1^ = (1, . . . , 1, 0, . . . , 0). The polynomial p (as well as P) is symmetrical 

k 

with respect to xi, . . . , Xm- It has the same value p{lk) = ^j=i(("^ — k)hj + 6^)2;^ for all 
permutations of 1^. Therefore, the set of inequalities should hold true 

> 



I 



together with the additional condition: 



J2{{m~l)bj + ej)Zj >0 
I 

^((m-2)6j- + 6^)2;^ > -a 



((m — + ej)zj > —a 



> -a 



m — 1 



m 

> -a| 2 I + a 



(M) There exist vectors {z^} t/iat twrn corresponding inequalities into equality. 

For any odd m > 2 we propose one of possible collections of variables hi, Cj that satisfy 
(fH?!) for Boolean variables (21, ... , Zm) as well as condition (M). The collection is identified 
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by the following system of equations 





- 2)h + ei) 




((m 


-3)6i + ei) 




((m 


-4)6i + ei) + ((m 


- 4)62 + 62) 


((m 


- 5)6i + ei) + ((m 


- 5)62 + 62) 



^ ((m - 2fc)6j- + Cj) 
E ((m-2fc-l)6,+e,) 



— ai 



—a 



'2' 
2 

2, 

2 

,2, 
2A; 



/2fc+ 1 

-a 

V 2 



—a 



m — 1 



m 

-a\ 2 ) + a 



(19) 



It is 



bi 



bi_i = 2a, hi = a, 



ej = -(2m - Aj + l)a, ( j = 1 - / - 1), 
Since the inequalities 



ei 



-2a. 



(20) 



(m — l)bj + Cj > 0, j = 1 ^ 

(m - 2A;)6j + < 0, {m-2k - l)bj + Cj < 0, if j<k, k = l^l-l, 
(m - 2A;)f)j + ej > 0, {m - 2k - l)bj + > 0, if j > A;, A; = 1 / - 1, 
6j + < 0, j = 1 -T- 1, 
ej < 0, J = 1 ^ /. 

hold true, the solution (pUj) actually satisfies inequality (fTHj) together with condition (M) 
for vectors z]; = Z2 = 0, Z3 = Z4 



For even m > 2 the collection of / 



m — 2 



+ 1 



variables 6,- , e,- can be found solution of 
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the system of equation 





- 2)h + ei) 




((m 


-3)6i + ei) 




((m 


-4)6i + ei) + ((m 


- 4)62 + 62) 


((m 


- 5)6i + ei) + ((m 


- 5)62 + 62) 



— ai 



—a 



^ ((m - 2fc)6j- + Cj) 
E ((m-2fc-l)6,+e,) 



E {2b, + e,) 



'2' 
2 

2, 

2 

,2, 
2A; 



/2fc+ 1 

—a 

V 2 



—a 



m — 2 
2 

m — 1 
2 



that looks similar to (|19|) but consists of only m equations, since the equation 



follows from the last two ones of system (j^D). The solution of the system is 

bj = 2a, Cj = —{2m — 4j + l)a, j = 1 I. 
Again the inequalities 

(m — l)bj + Cj > 0, j = 1 — I, 

(m - 2k)bj + ej < 0, {m-2k - l)bj + < 0, if j < k, k = 1 ^ I, 
(m - 2k)bj + Cj > 0, {m-2k - l)bj + > 0, if j > k, k = 1 ^ /, 



hold true and, therefore, (jl8j) together with condition (M) is fulfilled for vectors 



0, z 



' ^3 



li, 



^2fc+l ~ ^2fc+2 ~ Ifc) 



So, the coefficients of the polynomial p (see (fT7|) ) that represents P for odd and 
m are determined by ()20|) and ()22j). 

Lemmas ^1 and El allow to prove each P G Psm/ can be represented by a graph. 
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Theorem 14. The polynomials P G Vsuf o'^e graph representable. 

In order to proof the Theorem it is enough to use Lemma ^1 to represent nonhnear 
items of P with negative coefficients and, then, to exploit Lemma fTSl to determine that all 
items with positive coefficients can be accompanied by quadratic monomials and after be 
represented by graphs as well. 

Let the Boolean polynomial P{xi, . . . , Xn) G Vsuf of degree m consists of Lh monomials 
of degree more than two, then it can be represented by a graph with ^^^'^~^'> -)- n nodes. 
The worst case is when P{xi, . . . , Xn) G J^"^. The polynomial . . . , x^) G can be 

represented by a graph with Lh+n nodes. Hence, the global minimization of P{xi, . . . , Xn) G 
Vsuf require less than -)- n)^ operations, though for some classes of polynomials 

that occur in applications the number of operations can be reduced even toO(^^^^^ + n) 
(see, [H]). 



17 



References 

[I] Yu. Boykov, O. Veksler, R. Zabih. Fast Approximate Energy Minimization via Graph 
Cuts. IEEE Trans, on Pattern Anal, and Mach. Intel., 23(2001), No. 11, p.1222-1239. 

[2] S. Geman, D. Geman. Stochastic Relaxation, Gibbs Distributions, and the Bayesian 
Restoration of Images. IEEE Trans, on Pattern Anal, and Mach. Intel. PAMI-6, 6(1984), 
p.721-741. 

[3] D.M. Greig , B.T. Porteous, A.H. Seheult. Exact Maximum A Posteriori Estimation 
for Binary Images. J. R. Statist. Soc. B., 58(1989), p.271-279. 

[4] J. Buhmann, J. Puzicha. Basic principles of annealing for large scale nonlinear op- 
timization.. Gretschel, Martin (ed.) et al.. Online optimization of large scale systems. 
Berlin: Springer. (2001), p.749-777. 

[5] J. Buhmann, J. Puzicha. Multiscale annealing and robustness: Fast heuristics for large 
scale nonlinear optimization. (English) Gretschel, Martin (ed.) et al.. Online optimiza- 
tion of large scale systems. Berlin: Springer. (2001), p. 779-801. 

[6] H. Ishikawa. Exact Optimization for Markov Random Fields With Gonvex Priors. IEEE 
Trans, on Pattern Anal, and Mach. Intel. (To appear). 

[7] S. Iwata, L. Fleischer, S. Fujishige. A Combinatorial, Strongly Polynomial- Time Algo- 
rithm for Minimizing Submodular Functions. RIMS Kokyuroku, 1120(1999), p. 11-23. 

[8] V. Kolmogorov, R. Zabih. What Energy Functions Gan be Minimized via Graph Guts? 
Cornell CS Technical Report TR200 1-1857. 

[9] J.C. Picard, H.D. Ratliff. Minimal Guts and Related Problems. Networks, 5(1975), 
p.357-370. 

[10] B.A. Zalesky. Stochastic Relaxation for Building Some Glasses of Piecewise linear Re- 
gression Functions. Monte Carlo Methods and Appl., 6(2000), No. 2, p. 141-157. 

[II] B.A. Zalesky. Network Flow Optimization for Restoration of Images. J. of Applied 
Math., 2(2002), No.4, p. 199-218. http: / /jam.hindawi .com| 



18 



